- Title
- Artificial intelligence techniques to model the railway traffic management problem in tree topology railway networks
- Creator
- Messeder Caldas Bretas, Allan
- Relation
- University of Newcastle Research Higher Degree Thesis
- Resource Type
- thesis
- Date
- 2023
- Description
- Research Doctorate - Doctor of Philosophy (PhD)
- Description
- Rail freight planning problems pose specific challenges that have attracted the attention of academics and industry professionals for many decades. They involve multiple types of assets (trains, stations, terminals, etc), and are subjected to structural, operational and safety constraints. Even though a wide range of approaches have been proposed, few can address the complexity and size of real-world scenarios, and decentralised techniques, like multi-agent systems (MAS), have become more prevalent. The current state of the art in disciplines such as agent technology, reinforcement learning and discrete-event simulation allows the implementation of complex architectures, with multiple actors interacting and learning simultaneously. Therefore, this research takes advantage of these current advances and proposes innovative approaches to real-time traffic management problems in freight railway networks with a tree layout. This study was motivated by the decision-making scheduling problems arising in the Hunter Valley Coal Chain (HVCC), located in New South Wales, Australia. First, a simplified simulation model was implemented to experiment with reinforcement learning (RL) in a centralised fashion, which achieved relevant results but also revealed some limitations. Then, a new decentralised framework was developed to reach the features of the real problem. The approach uses the simulation model currently utilised for capacity planning for the HVCC as the training environment, facilitating experiments with actual data. The simulation model was enhanced to accommodate a MAS with intelligent agents representing system elements, such as trains, dump stations, and load points. Furthermore, these agents act in a decentralised fashion based on local observations, constituting a partially observed Markov decision process. Two main decision methods were applied to guide the agents' decisions: genetic algorithms (GA) and RL, and three variations of the multi-agent deep reinforcement learning (MADRL) approaches were developed: a baseline model, an extended model, and one that directly addresses deadlocks. Finally, the learned policies were tested in a transfer learning strategy to cope with unseen deadlocks. The experiments explore specific complex scenarios arising in the HVCC, where trains frequently face deadlock conditions. The MADRL baseline model provides lower dwell times than a first-come-first-served (FCFS) based heuristic in use by HVCC's simulation model and the GA in instances with up 60 trains -- but fails in more complex scenarios. On the other hand, the most advanced model always finds feasible solutions. In addition, it outperforms the FCFS-based heuristic by up to 53% in instances with up to 128 trains. The framework represents the main outcome of this research. The model embodies a novel application of MAS to rail freight traffic management in tree-layout railway networks, and fulfils the need for decentralised and scalable methods. The agents (a) learn how to avoid deadlocks, (b) deal with different occupation levels, (c) predict conflicts, (d) manage different track lengths and (e) avoid long loading/unloading queues. These are significant achievements for the joint research between HVCC and the University of Newcastle, and open new possibilities for evaluating the rail network capacity and increasing throughput.
- Subject
- simulation-based machine learning; railway traffic management; multi-agent deep reinforcement learning; genetic algorithms
- Identifier
- http://hdl.handle.net/1959.13/1507189
- Identifier
- uon:55986
- Rights
- Copyright 2023 Allan Messeder Caldas Bretas
- Language
- eng
- Full Text
- Hits: 208
- Visitors: 246
- Downloads: 45
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | ATTACHMENT01 | Thesis | 4 MB | Adobe Acrobat PDF | View Details Download | ||
| View Details Download | ATTACHMENT02 | Abstract | 211 KB | Adobe Acrobat PDF | View Details Download |